1. Field of the Invention
This invention relates to computers and digital processing systems. More particularly, this invention relates to storage system management and control.
2. Description of the Related Art
The meanings of certain acronyms and abbreviations used herein are given in Table 1.
TABLE 1Acronyms and AbbreviationsCPUCentral Processing UnitDMADirect Memory AccessGPUGraphics Processing UnitI/OInput/OutputIDIdentifierLBALogical Block AddressNICNetwork Interface CardOSOperating SystemPCIPeripheral Component InterconnectPCIePeripheral Component Interconnect ExpressPFPhysical FunctionPLBAPhysical Logical Block AddressSR-IOVSingle Root I/O VirtualizationTLBTranslation Lookaside BufferVFVirtual FunctionVLBAVirtual Logical Block AddressVMVirtual Machine
Emerging high-performance data processing systems make use of multiple processors, accelerators and various physical devices and compartmentalize applications inside virtual machines. These rich environments are composed of many physical and virtual entities that need fast access to disk storage. However, the increasing complexity and versatility of these systems require that they provide strict data security and isolation mechanisms that can protect data from being accessed by an unauthorized entity.
Traditionally, security and performance have been considered opposing trends; providing data security typically incurs performance overheads (and vice versa). For example, existing systems provide security guarantees by requiring that only a single trusted entity, namely the operating system, can access data stored on disks. The operating system enforces security policies and acts as a proxy for all other system entities. As a result, all data accesses incur a substantial overhead when marshaling requests to the disk controllers through the operating system. However, the prevalent single-trusted-entity only emphasizes the data access overheads for virtual machines and physical accelerators (as well as other devices in the system), for orthogonal reasons. On one hand, virtual machines employ an internal operating system to provide data protection for their internally executing applications, thus replicating the data access overhead. On the other hand, physical accelerators and other devices can directly access the peripheral device interconnect and disk controllers attached to it. However, since data protection and isolation are provided by the operating system, they must communicate with the disk controllers by forwarding all their data requests through the central processing unit (CPU), which runs the operating system.
Generally, accelerators and other devices in a system rely on the CPU cooperatively with the operating system to arbitrate data access requests involving storage units. Indeed, the CPU itself may formulate and transmit such requests. In the case of a virtual environment, storage I/O is usually managed by the hypervisor, which must multiplex outgoing I/O requests from several virtual machines to a single storage device. There are several methods of I/O virtualization:
Emulation: The hypervisor traps I/O requests from the virtual machine (VM) and emulates the behavior of a storage device with a file in its file system.
Paravirtualization: The VM is aware that it is running on a virtual machine and its drivers communicate with the underlying hypervisor directly, which also emulates the storage.
Direct I/O: The hypervisor gains access to the address space of the device on behalf of the VM, and the VM then communicates directly with the device without the hypervisor's intervention.
In emulation and paravirtualization security and isolation are achieved by the hypervisor. Every storage device is emulated by a file on its file system, and when a VM requests to access its storage device, the hypervisor routes the request to the correct offset in that file. In these methods, there is a performance overhead because every request must be taken care of by the hypervisor. If the storage device is shared by another accelerator in the system, for example in a graphics processing unit (GPU) or another device, every request from the accelerator must also go through the host OS and the hypervisor, which prevents the host CPU from being idle. In this case, there is an impact on the power consumption of the host CPU.
In the case of direct I/O, security is enforced at the storage device level because only one VM can control the storage device. This method has almost no performance overhead, but has the drawback of not being able to share the device among several VM's.